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Abstract. In decision problems, often, utilities and probabilities are hard to determine. In 
such cases, one can resort to so-called choice functions. They provide a means to determine 
which options in a particular set are optimal, and allow incomparability among any number of 
options. Applying choice functions in sequential decision problems can be highly non-trivial, as 
the usual properties of maximizing expected utility may no longer be satisfied. In this paper, 
we study one of these properties: we revisit and reinterpret Selten's concept of subgame per- 
fectness in the context of decision trees, leading us to the concept of subtree perfectness, which 
basically says that the optimal solution of a decision tree should not depend on any larger tree 
it may be embedded in. In other words, subtree perfectness excludes counterfactual reasoning, 
and therefore may be desirable from some philosophical points of view. Subtree perfectness is 
also desirable from a practical point of view, because it admits efficient algorithms for solving 
decision trees, such as backward induction. The main contribution of this paper is a very 
simple non-technical criterion for determining whether any given choice function will satisfy 
subtree perfectness or not. We demonstrate the theorem and illustrate subtree perfectness, 
or the lack thereof, through numerous examples, for a wide variety of choice functions, where 
incomparability among strategies can be caused by imprecision in either probabilities or util- 
ities. We find that almost no choice function, except for maximizing expected utility, satisfies 
it in general. We also find that choice functions other than maximizing expected utility can 
satisfy it, provided that we restrict either the structure of the tree, or the structure of the 
choice function. 



1. Introduction 

In statistical decision problems, one is often faced with too little data for too many parameters, 
and elicitation of all probabilities and utilities can be a prohibitively expensive process, rendering 
full modeling infeasible. In such cases, it may still be possible to bound the set of reasonable 
models, i.e. to bound probability distributions and utility functions [31 UHl HZ1 HS1 Ell HI] • The 
way in which such models can be used for decision making is well studied [5S1 [TTJ []] [31 [2H HH 

na 123112a Ha- 

In this paper, we consider a very large class of decision models, namely any that can be 
represented by conditional choice functions on gambles: we merely assume that for any finite set 
of gambles (functions from the possibility space f2 to a set of rewards 1Z; these generalize random 
variables, or horse lotteries), and any conditioning event, the subject can give a non-empty subset 
of gambles that he considers optimal if the conditioning event were to occur. Gambles that are 
non-optimal would never be selected, and the subject is unable to express further preference 
between the optimal ones. Maximizing conditional expected utility is a simple example of such 
a choice function. General choice functions, however, need neither probability nor utility — not 
even a total preorder, and in fact, not even a partial one, although obviously these obtain as 
special cases. 

Then, in a single agent sequential decision problem, at any stage, one has two ways of looking 
at its solution: the problem can be considered either in its simplest form — discarding any past 
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Figure 1. Two-stage problem. Figure 2. Second stage. 



stages, or as part of a much larger problem — possibly considering choices and events that did not 
actually obtain. A reasonable requirement is that, at any stage, the solution is independent of 
the larger problem it is embedded in. In this paper, we call this requirement subtree perfectness. 

Selten |24j introduced a similar idea for multi-agent extensive form games, called subgame 
perfectness. Solutions of extensive form games take the form of equilibrium points. If, for every 
subgame (a part of a game that is again a game), the restriction of the equilibrium point of the full 
game to that subgame also yields an equilibrium point of that subgame, then the equilibrium 
point of the full game is called subgame perfect. Selten showed that for games with perfect 
recall, (perfeclQ) equilibrium points are subgame perfect [Ml P- 39, Thm. 2], that is, they are 
independent of any larger game in which they could be embedded. 

In this paper we investigate single agent sequential decision making modeled by decision trees. 
Although such problems differ in many ways from extensive form games, subtree perfectness is 
clearly analogous to subgame perfectness, as the following example shows. 

Consider the decision problem in Fig. [1] (replicated from [5]). In the first stage, the subject 
chooses between taking scones, or proceeding to the second stage. In the second stage, the 
subject chooses between cake, or ice cream. Suppose the subject prefers to reject scones and to 
choose ice cream at the second stage. This strategy induces a substrategy in the subtree for the 
second stage: choose ice cream over cake. 

But, as with multi-agent games, we can instead consider the subtree for the second stage 
separately, as in Fig. [5J If, in this smaller tree, the subject prefers ice cream, then his solution is 
subtree perfect: his solution for the full tree induces a strategy in the subtree, and this strategy 
coincides with his solution for the subtree. If the subject states a different preference (either no 
preference, or clear preference for cake), then his solution lacks subtree perfectness. So, subtree 
perfectness essentially means that the optimal induced strategies in a subtree do not depend on 
the full tree in which the subtree is embedded. 

Our main result, given in Theorem [2] further in the paper, is an extremely straightforward 
necessary and sufficient condition on choice functions for subtree perfectness to hold in any 
decision tree. A similar result was given rigorous treatment in [9], however that treatment was 
rather technical and opaque, and not as obvious to apply to practical problems. The main 
contribution of this paper is to provide an overview of the result that is accessible and easy to 
apply, but still general enough to apply to any choice function. Therefore, there is far more 
emphasis on rigorous examples, diagrams, and procedures, and far less on detailed definitions 
and lengthy proofs. In so doing, we establish a useful philosophical and practical link between 
the work that has been done on the behavior of specific choice functions, such as [TTJ[21[32], the 
more general and theoretical works such as [SJ [TTj HH1 H] , and proposed algorithms for solving 
decision trees [5J HH HI HI EH EH] • Readers seeking formal proofs of the results, or those who are 
interested in proving similar results about sequential decision problems, are directed to [5]. 



Without going into much detail, a perfect equilibrium point is one which is stable under small perturbations 
[H p. 38], 
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In this paper we consider only the traditional normal form method of listing all strategies, 
finding their corresponding gambles, applying a choice function, and listing all strategies that 
induce optimal gambles. In doing so, we assume act-state independence — that is, choice functions 
do not depend on the decision. This type of solution is also called resolute [HI S3]- Several 
related works have considered the alternative extensive form solutions, such as in Hammond [5] , 
Seidenfeld |21j . and Machina [T7]. We do not consider extensive form solutions in this paper, 
although the results are closely related. Without going into details, subtree perfectness is similar 
to Hammond's consistency and consequentialism, Machina's separability over mutually exclusive 
events, and McClennen's separability and dynamic consistency 18 . Further details about the 
links with extensive form subtree perfectness can be found in [9]. Subtree perfectness also is 
related to the weaker concept of backward induction. More details on normal form backward 
induction can be found in |TU] . 

It should also be noted that the algorithm proposed by Kikuti et al. [HJ[T3] follows exactly our 
concept of normal form backward induction, although they then use it to construct an extensive 
form (what they call consequentialist) solution. They note that, for all the choice functions 
they consider, their solution is "inconsistent" with our normal form solution, which essentially 
means that subtree perfectness is violated. For two these choice functions, maximality and 
E-admissibility, it is observed that this inconsistency is not so severe: although arcs that are 
non-optimal in the full tree may appear in local trees, no arc that is non-optimal in a local tree 
may reappear in the full tree. In Fig.[TJ for example, it would not be possible under these criteria 
for the subject to say they preferred cake to ice cream at AT 2 and then at N\ include ice cream in 
the optimal set. This weaker form of subtree perfectness occurs because the two criteria satisfy 
the conditions of our backward induction theorem [10] . What this means in practice is that for 
these criteria the work of Kikuti et al. can be used to efficiently construct the consequentialist 
solution that they are interested in, and also the normal form solution that we are interested 
in, and although subtree perfectness does not hold, both the solutions are at least somewhat 
consistent and sensible. 

The paper is structured as follows. Section 12.11 provides an introductory example using ex- 
pected utility to outline what we mean by a solution of a tree, and what we mean by subtree 
perfectness. Sections 12 .21 and l2~31 give rigorous, yet non-technical, definitions of the basic concepts 
required to formulate our main result. That result, the subtree perfectness theorem, is provided 
in Section 12.41 We then demonstrate the theorem on numerous choice functions and decision 
trees: Section [3.11 gives further examples of satisfaction of subtree perfectness for certain choice 
functions and trees, whilst Section 13.21 gives further examples of failures of subtree perfectness. 
Section U] concludes the paper. 

2. Subtree Perfectness 

2.1. Examples: Expected Utility. We begin by considering some examples of solving small 
decision trees using expected utility. Of course, these solutions are almost trivial to find, but we 
spend some time on them to outline our ideas in a familiar environment. Consider the decision 
tree in Fig. |3l and suppose that P(Ei) — 0.6, P(£ , 2) = 0.4. Square decision nodes are points 
where the subject chooses the arc to follow. Round chance nodes are points where the arc 
followed depends on the (initially unknown) state of nature, in this case E\ or E2. Typically one 
would approach the problem by backward induction |16j , but we consider an alternative method. 
This is because for more general choice functions, backward induction is more complicated and 
may not give the desired answer. 

Consider a path that the subject may take through the tree. At N , she has a choice of arcs 
and suppose she follows the arc to N\. Here again she has a choice; suppose she takes the arc to 
N%i. Here, she has no choice over the arc followed: nature decides. Thus, for this combination 
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FIGURE 3. Decision tree for first expected utility example. 



N 




FIGURE 4. Normal form solution for first expected utility example. 

of her choices, her final reward is 2 if E\ occurs and otherwise. We can calculate the expected 
utility of this strategy to be 1.2. 

There are three other strategies in the tree: the paths N — > N\ — > N12, N — > N2 — > N21, 
and JV — » N2 — > N22- These have expected utilities 1, 1.8, and 2 respectively. We therefore 
conclude that the solution of the tree is to take the path from N to N2 and then to A^2- This 
is illustrated in the decision tree in Fig. 2J 

Now, subtree perfectness is a property relating solutions of large trees to solutions of subtrees. 
In this particular example, there are two non-trivial subtrees of the full tree, shown in Fig. [5l 
We can solve these trees in the same manner; indeed we have already calculated the required 
expected utilities. The optimal paths are N\ — > Nn in the left-hand tree, and N2 —> N22 in the 
right-hand tree. These solutions are displayed in Fig[B] 

Are these local solutions consistent with the solution in Fig. H]? The first thing to note is 
that Ni does not appear at all in Fig. |U We can therefore ignore it: behavior in a subtree that 
will never actually be reached is irrelevant to subtree perfectness. At N%, we see that both the 
solutions (Fig. 2] and Fig. [5]) contain only the path to A^2, so there is no inconsistency here 
either. We say that subtree perfectness holds for this problem. 
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FIGURE 5. Subtrees for first expected utility example. 




Figure 6. Solutions of subtrees for first expected utility example. 




FIGURE 7. Decision tree for second and third expected utility examples. 

Next we move onto a slightly more involved example, in which the initial node is a chance 
node, as shown in Fig. [71 We need a few more probabilities for this example: let P(E\\Ai) = 0.6, 
P(E 2 \A 1 ) = 0.4, P(Et\A 2 ) = 0.4, P{E 2 \A 2 ) = 0.6, P{A X ) = 0.5, and P(A 2 ) = 0.5. 

In this example, the two available strategies are "if At, take the path iVi — > Ntt", and "if 
A\, take the path Ni — > Ni 2 " . Note that there is no decision node following A 2 , so the subject 
does not need to specify anything in that case. These strategies have slightly more complicated 
gambles than before. Consider the first: if A\ and E\ both occur, the outcome is 2; if A\ and E 2 
both occur, the outcome is 0; if A 2 and E\ both occur, the outcome is 3; and if A 2 and E 2 both 
occur, the outcome is 0. This gives expected utility 0.5(0.6 • 2 + 0.4 • 0) + 0.5(0.4 • 3 + 0.6 • 0) = 1.2. 
The second strategy gives expected utility 0.5 ■ 1 + 0.5(0.4 -3 + 0.6-0) = 1.1. Hence the solution 
is the first strategy, shown in Fig [5] 
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Figure 8. Normal form solution for second expected utility example. 




Figure 9. Subtree for second expected utility example. 



Again, we look for non-trivial subtrees. In this example, there is only one, at Nx. This is 
shown in Fig [9] — note that this is exactly the left-hand tree of Fig. [5j We can solve this as usual, 
although here we have to remember that the event A\ has occurred, and so we must use the 
suitable conditional probabilities. The expected utilities of the two strategies in this case are 
0.6 • 2 + 0.4 -0 = 1.2 and 1 respectively, so the normal form solution of this subtree is to take the 
path Ni — > Nn (the same as the left-hand tree in Fig- EJ) - 

Once again we check for subtree perfectness. In both the global and the local solutions, the 
only path from Ni is the one to Nix, hence subtree perfectness is satisfied in this example. 

Finally, we provide an example of expected utility failing subtree perfectness. As we show 
later, this requires a conditioning event with probability zero. Changing the probabilities in the 
previous example to P{Ai) = 0, f (^2) = 1 will be sufficient to fail subtree perfectness. Now the 
expected utilities of the two strategies in the tree of Fig.[7]are 0(0. 6-2 + 0. 4-0) + l(0.4-3 + 0. 6-0) = 
1.2 and • 1 + 1(0.4 • 3 + 0.6 • 0) = 1.2 respectively. Hence, we cannot choose between the 
two strategies using expected utility, and the normal form solution must include both. This is 
illustrated in Fig [TO] 

Because we did not change any of the conditional probabilities, the normal form solution of 
the subtree at Nx will be the same as in the previous example: the left-hand tree of Fig. [51 
Now we see inconsistency between the two solutions: one of the strategies in the global solution 
contains the path Nx — > Nx2, but this path does not appear in the local solution. We say that 
subtree perfectness does not hold at Nx- 
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Figure 10. Normal form solution for third expected utility example. 

Having seen how subtree perfectness works for a standard choice function and simple trees, 
we now explain how to carry out the steps above for any tree and any choice function. 

2.2. Definition of Subtree Perfectness. To state our main result, we need to define sub- 
tree perfectness for arbitrarily large (but finite) decision trees. In this paper, we will describe 
and define the concept of subtree perfectness directly on decision trees, without further formal 
mathematical notation. The tree in Fig. [TT] will serve as a leading example. Of course, subtree 
perfectness can also be defined more formally, using mathematical notation that is very useful 
for proofs, as we did in [5], however such approach is rather technical, and unnecessary for illus- 
trating the main ideas. Any reader who is interested in rigorous mathematical formulations and 
proofs of properties relating to choice functions and decision trees is referred to [9]. It should 
also be noted that, although throughout this paper almost all our examples feature real-valued 
rewards, the language and results we introduce here will work for any form of rewards. 

The solutions we consider are sets of strategies. A strategy corresponds to the subject initially 
specifying all her actions in all eventualities, that is, an action for each decision node she may 
possibly reach. Upon arriving at a decision node, she then follows the mandated action. Thus, 
after the strategy is chosen, the subject no longer has any control. In other words, she is a 
resolute decision maker [18 . 

A strategy is easy to represent as a subtree of the initial tree. Consider the decision tree in 
Fig. QTJ and the strategy "at Ni choose ds, then if Si occurs choose d\ and if S2 occurs choose 
c^". This can be represented by the decision tree in Fig. 1121 In this figure, an arc with a double 
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Figure 11. A more complex example of a decision tree. 




Figure 12. One of the strategies of the tree in Fig. [TTJ 

line through means the arc has been deleted. So the decision tree representation of a strategy is 
the original tree minus all decision arcs that are not in the strategy. Note that the labels of the 
nodes were retained in this process: this will be an important point later on. Also note that this 
is now a trivial decision tree, since there is only one option available at each decision node. 
In general, we can construct a strategy for a given decision tree by the following method: 

(1) For each decision node, pick exactly one decision arc and delete all others. 

(2) Delete everything not connected to the original root node of the tree. 

By applying this method for all possible combinations of choices of decision arcs, we arrive at 
the set of all available strategies for a given decision tree0 

2 Of course, in practice, there is a more efficient way to find all strategies — for instance, in Fig. 1111 there are 
2 4 = 16 combinations of arc choices, but these lead to only 6 distinct strategies: evidently, one does not need to 
iterate over arcs of decision nodes in subtrees following deleted arcs. 




As in the examples earlier, we consider solutions of decision trees of the following type: first 
we find all available strategies, then we apply some criterion to them that returns a non-empty 
subset of strategies. Ideally, we might want our criterion to return a single strategy. However, 
as argued in the introduction, in many situations the subject may not have enough information 
to make such a judgment, for instance because their probabilities and utilities are not fully 
determined. A function that is applied to a set of options to return a non-empty subset is called 
a choice function. The set of strategies that a choice function returns is called a normal form 
solution of the decision tree. So in the case of Fig. 1111 we would first find the six strategies and 
then apply our choice function to these. Let us suppose that a particular choice function gives 
the normal form solution in Fig. [13] Note that here, we apply the choice function conditional on 
the certain event. Conditioning on other events is only relevant for subtrees, discussed next. 

To examine subtree pcrfcctncss, we need to know how to restrict trees to smaller subtrees and 
solve these subtrees, and also how to restrict the solutions of larger trees to smaller subtrees. 
The first task is very straightforward: for each node on a tree, the subtree at that node is the 
node itself and all of its descendants. For example, the subtree at Nm in Fig. [IT] is shown in 
Fig. HH Observe that the labels of the nodes are as in the original tree. Then the final step of 
the procedure is simply to solve this subtree, conditional on the logical conjunction of all events 
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Figure 14. The subtree of Fig. [TT1 at Nm. 




Figure 15. Possible normal form solution of Fig. [TH 

leading up to Nm (or, conditional on the certain event, if no chance nodes are visited) H In our 
example, we find the two normal form solutions in Fig. fT?] and then apply our choice function, 
conditional on Si (this is the only event leading up to Nm), to this set. Let us suppose this 
gives the singleton strategy in Fig. [15] 

This process of restriction and then solution can be summarized as follows. 

(1) Choose a node N to which to restrict the tree T. 

(2) Find the conjunction of events which lead up to N from the root of T. Denote this 
conjunction by E. If there are no events leading up to N, then take E to be the certain 
event. 

(3) Remove from T all non-descendants of N other than N itself. This is the subtree at N. 

(4) Find all strategies for this subtree. 

(5) Apply the choice function to this set of strategies, conditional on E. This is the normal 
form solution of the restricted tree. 

The opposite order, solving the large tree and then restricting the solution, is a little more 
involved. We saw the first step in Fig. [T3l How can this solution be restricted to, say, Nm? 
This is done element-by-element. We take the first strategy in the solution, and restrict it to 
Nm exactly as above. This gives the same tree as in Fig. rj5J We then attempt to restrict the 
second strategy to Nm, but we notice that this node does not appear in the strategy. In such a 
case, we remove the strategy entirely. Next we move to the third strategy, but again Nm is not 
present, so remove this strategy too. This is the final strategy in the solution, so we find that 
the restriction of the normal form solution to Nm is the singleton in Fig. 1151 In this example, 
we see that restricting the tree to Ni 1 1 and then solving is the same as solving the full tree and 
restricting the solution to Nm. We say that subtree perfectness is satisfied at iVm. 

In general, the process for solving a tree and then restricting the solution to a subtree goes as 
follows. 

(1) Find all strategies for the tree. 

(2) Apply the choice function to this set of strategies. This is the normal form solution of 
the tree. 



We exclude decision trees that have for the conditioning event of any of their subtrees, as most theories 
of conditioning do not allow conditioning on the empty set. This goes without loss of generality, because any 
decision tree can be 'fixed' simply by pruning its 0-conditioncd subtrees. 
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(3) Choose a node N to restrict the solution to. 

(4) For each strategy T" in the solution: 

• If ./V is not a node of T", remove T 1 . 

• Otherwise, remove from T" all non-descendants of N other than N itself. 

(5) This gives the restriction of the normal form solution. 

Now we have all the tools to define subtree perfectness. 

Definition 1 (Subtree Perfectness). Let a decision tree T , a node N in T , and a choice function 
be given. Carry out the operations of restricting the solution ofT to N , and solving the restriction 
of T to N . The choice function is called subtree perfect for T at N if either 

• the restriction of the solution is empty (that is, N is not part of any strategy in the 
solution of T ); or 

• the restriction of the solution equals the solution of the restriction. 

The choice function is called subtree perfect for T if it is subtree perfect at all nodes N in T . 
The choice function is called subtree perfect if it is subtree perfect for every T . 

Subtree perfect solutions are useful for several reasons. When solving a decision problem, one 
can usually see it as being embedded in some larger decision problem that started in the past. 
With a subtree perfect solution, the rest of this larger problem can be ignored. A choice function 
that fails subtree perfectness may require consideration of events that did not happen or options 
that were refused in the past. This is practically and perhaps intuitively unappealing. Similarly, 
if the decision tree changes midway through the problem (for instance, new options that were 
not modeled become available), lack of subtree perfectness may require considering the whole 
problem again, whereas with subtree perfectness only the local problem needs to be solved. 

2.3. Choice Functions on Gambles. The procedure introduced for finding a normal form 
solution involved applying a choice function to the set of available strategies. In practice, it is 
rare to apply choice functions to sets of strategies, but rather to sets of gambles. A gamble is a 
map from the set of possible outcomes to a set of rewards — to avoid technicalities, we assume 
the set of possible outcomes to be finite. 

For example, consider the three strategies in Fig. [13] The first of these will give 9 if S\ n E\ 
occurs, 14 if S\ fl E 2 occurs, 4 if 52 n Ex occurs, and 19 if 52 n E 2 occurs. This covers all possible 
outcomes, so for any outcome we know what reward the subject will receive. This is exactly 
what we call a gamble. 

We write such a gamble 95i^i + 145iS 2 + 452 £i + 195 2 £ 2 , or even as 5i(9£'i + IAE 2 ) + 
S 2 (4:Ei+19E 2 ). Similarly, the other two strategies in Fig. ITBIcorrespond to the gambles 5i(10-E'i+ 
15E 2 ) + 5 2 (10£i + 15E 2 ) and 5i(5£i + 20E 2 ) + S 2 {5E! + 20E 2 ) respectively. Note that which 
one of 5i and 52 occurs is irrelevant to both of these gambles, so they could be written as 
10-Ei + 15E 2 and 5Ei + 20E 2 respectively. These objects are much more natural to work with; 
for instance, should we know the probabilities of the four events, then we could calculate the 
expected utility of the three gambles. To work with sequential decision making, we actually need 
conditional choice functions: the set returned by the choice function depends on what event is 
being conditioned upon. 

Many, if not all, common choice functions used in decision theory are choice functions on 
gambles, hence restricting attention to them makes sense. This adds two extra steps to the 
procedure for finding a normal form solution: 

(1) Find all available strategies. 

(2) For each strategy, find its corresponding gamble. 

(3) Apply the choice function to this set of gambles. 
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(A) (B) 



Figure 16. Decision trees for which subtree perfectness implies subtree per- 
fectness for any other tree. 

(4) For each gamble chosen by the choice function, find all strategies in the original set that 
induced this gamble. This is the normal form solution. In other words, a strategy is in 
the normal form solution if and only if its corresponding gamble is chosen by the choice 
function. 

Note that we lose some generality by moving to choice functions on gambles, because there may 
be several strategies with the same corresponding gamble. A choice function on gambles would 
be unable to distinguish between these, but a choice function on strategies would. This loss is 
not so important, given that it would be rare to want to distinguish between strategies that give 
the same reward in every eventuality. 

2.4. Subtree Perfectness Theorem. Now we can present our main result. We are interested 
in when a particular choice function on gambles is subtree perfect. It turns out that all that is 
required for this is subtree perfectness for two simple classes of decision trees, shown in Fig. 1161 
In that figure, the Xi, and Z are arbitrary gambles. As part of a tree, they represent a 
terminal chance node. For instance, the gamble X = bE\ + 3i?2 would represent a chance node 
with two arcs, with event E\ leading to reward 5, and event Ei leading to reward 3. 

Theorem 2 (Subtree Perfectness Theorem). A choice function on gambles is subtree perfect 
if and only if it is subtree perfect for all decision trees of the two types in Fig. [731 f or every 
conditioning event B at N\ such that A n B ^ and A c C\ B =/= 0, and for every non-empty 
conditioning event B at N2 ■ 

So, in Fig. 1161 we consider solving the full trees conditional not just on the certain event, but 
conditional on every possible non-empty event that does not cause the decision tree to have a 
path corresponding to 0. 

This theorem is equivalent to the subtree perfectness theorem presented and proved in [SJ 
Theorem 22], but is perhaps more straightforward to understand. All we need to check for 
subtree perfectness is two simple types of decision tree, and this proves very useful when seeking 
both proofs and counterexamples for a particular choice function. Note that it is not immediately 
obvious that this theorem is equivalent to [9j Theorem 22], but in fact it appears in the proof [9l 
A.9]. 

3. Examples 

3.1. Examples of Subtree Perfectness. In this section we give some examples of choice 
functions and decision trees where subtree perfectness holds. We concentrate on two simple 
trees, shown in Fig. 1161 This is not just for simplicity; as we saw in Section [21 these two types 
of trees are key to subtree perfectness. 
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3.1.1. Expected Utility. 

Example 3. First, we consider the simplest case, and investigate the tree in Fig. \l^(b)\ with 
conditioning event B. Maximizing conditional expectation for this tree is always subtree perfect, 
simply because 

(1) ma3L{E{Xx\B), . . . ,E{X n \B), E{Y 1 \B), . . . , E(Y m \B)} 

= ma X {ma X {E(X 1 \B), . . . , E(X n \B)}, max{£(Yi|fl), . . . , E(Y m \B)}}. 



Example 4. Consider the tree in Fig. \l£f(a)\ with conditioning event B such that A n B ^ 
and A c PI B ^ 0. On the one hand, the normal form solution of the full tree is obtained by those 
gambles AXi + A C Z which maximize their expectation conditional on B: 

(2) &rgm&xE(AXi + A C Z\B). 

i—l 

On the other hand, the normal form solution of the subtree at N±i is obtained by those gambles 
Xi which maximize their expectation conditional on A (1 B: 

(3) BXgTQax.E{Xi\AnB) 

i—l 

Now, note that, by the partition theorem, 

(4) E(AX t + A C Z\B) = P(A\B)E(Xi\A n B) + P(A C \B)E(Z\A C n B). 

Because the term P(A C \B)E{Z\A C n B) is independent of the index i, the solution of both maxi- 
mization problems is identical: 

(5) BXgTuax.E{AXi+ A C Z\B) = arg max E(Xi \Ar\B), 

i—l i—l 

provided that P(A\B) > 0. In other words, maximizing expected utility satisfies subtree perfect- 
ness on this type of tree whenever the probability of every non-empty event is strictly positive. 

From the above analysis, and Theorem [5J we recover a well-known result: expected utility 
satisfies subtree perfectness for arbitrary decision trees, provided that all chance arcs have strictly 
positive probability. Interestingly, if some chance arcs have probability zero, subtree perfectness 
may fail; we already saw an instance of such failure in the last example of Section 12.11 

3.1.2. Choice Functions Induced By A Total Preorder. In Example [3j where we established sub- 
tree perfectness of maximizing expectation for the tree in Fig. Il(|^b)[ all we really relied on is 
the fact that expectation induces a total preorder over gambles. 

Let us be a bit more specific. Consider a relation y over gambles, which satisfies transitivity, 
reflexivity, and completeness; that is, for all gambles X, Y , and Z, we require that: 

(1) X h Y and Y > Z implies X > Z; 

(2) X h X; and 

(3) X t Y or Y t X. 

Any relation which satisfies these properties is called a total preorder. Now, an obvious way to 
associate a choice function with a total preorder is to select those gambles which dominate all 
other gambles: 

(6) m&xX := {X E X: MY E X,X h Y}, 

for any finite set of gambles X. Now, it is easily established that, for any total preorder ^ over 
gambles, 

(7) max{Xx, X n , Y x , . . . , Y m } = max{max{Xi, . . . , X n }, max{Yi, . . . , Y m }}. 
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In other words, any choice function induced by a total preorder is subtree perfect for trees of the 
type of the tree in Fig. [T^b)[ 

In fact, a much stronger result holds: a choice function is subtree perfect only if it is induced 
by a total preorder [9j Theorem 22 & Lemma 15]. Consequently, many of the choice functions 
used for imprecise probability and imprecise utility fail subtree perfectness, as they are typically 
not induced by total preorders; specifically, they typically drop completeness. Interestingly, 
completeness is not required for some schemes of backward induction to work [TOJ [5J [T31 [TO] ■ 

3.1.3. Maximin. The maximin criterion values a gamble at its smallest possible reward, and then 
chooses gambles that give the maximum value. Because it is a total preorder, subtree perfectness 
will hold for the tree in Fig. Il(^b)[ as seen in Section 13.1.21 

Maximin does not in general hold for the tree in Fig.ll(|ta)| we shall see this later in Example[9l 



3. 1 .4. r -Maximin. Suppose that too little information is available to identify a unique probability 
distribution, but we can specify a set of plausible probability distributions (see for instance 
[S [IB E3I2H1 III])- In particular, suppose that 

• rewards are expressed in utiles, so 1Z = R, 

• the subject can express her beliefs by means of a set M. of probability distributions P 
(A4 is called the credal set, and is typically assumed to be closed and convex, however 
we need no such assumption here), and 

• each probability distribution P 6 M. satisfies P(A) > for all events A ^ 0. 

• Note that the subject does not express beliefs about the relative plausibility of the distri- 
butions, so there is no second-order probability over the set. Also note that the positive 
probabilities are not necessary to define imprecise probability, but this assumption lets 
us avoid some technical details. 

Under the above assumptions, each P in M. determines a conditional expectation^ 



E P (X\A) 



P(A) 

and the whole set A4 determines a conditional lower and upper expectation 

(8) MX\A) = min E P (X\A) E(X\A) = max E P (X\A), 

P £M PGM 

and this for every gamble X and every non-empty event A. We consider four choice functions for 
this uncertainty model. Each is introduced only briefly; see [25] for a more in-depth discussion 
and comparison. 

The first, T- maximin, selects any gamble that maximizes lower expectation E_. Failure of 
subtree perfectness for r-maximin has been well-documented [211 22} , but for lower expectations 
with a particular structure, subtree perfectness can hold. 



Example 5. Consider the tree in Fig. \l^a)\ for simplicity of notation, we assume here that 
the conditioning event B is the certain event, however the analysis extends trivially to any con- 
ditioning event. 

Suppose also that E_ satisfies marginal extension [261 §6.7.2] , that is, 
(9) E{AX + A C Y) = E(AE{X\A) + A C E(Y\A C )). 

for any gambles X and Y. Note that this is naturally satisfied if beliefs are expressed as an 
unconditional lower expectation over gambles of the form Act + A c f3 (for arbitrary a, /3 £ Rj, 
and conditional lower expectations E_(-\A) and E_(-\A C ). 



In this paper, gambles, such as X, are always assumed to take only a finite number of values. 
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On the one hand, the normal form solution of the full tree is obtained by those gambles AXi + 
A C Z which maximize their lower expectation: 

(10) arg max £( AX* + A C Z). 

i—l 

On the other hand, the normal form solution of the subtree at N±i is obtained by those gambles 
Xi which maximize their lower expectation conditional on A: 

(11) argmaxE(Xi|A) 

2—1 

Now, note that, by marginal extension, 

(12) E(AXi + A C Z) = E{AE{Xi\A) + A C E{Z\A C )). 

Using the above identity, one can easily show that the solution of both maximization problems is 
identical: 

(13) arg maxE(AXi + A C Z) = arg maxfiffi \A) 

z—l i—l 

provided that E_(A) > 0. In other words, T-maximin satisfies subtree perfectness on this type of 
tree whenever the lower probability of every event is strictly positive. 

Clearly, r-maximin corresponds to a total preorder, and therefore, it will also satisfy subtree 
perfectness for the tree in Fig. Il(|^b)[ Concluding, as long as the lower probability of every event 
is strictly positive, and marginal extension is satisfied with respect to the way the chance nodes 
partition the possibility space, r-maximin satisfies subtree perfectness. 



3.1.5. T-Maximax. The next criterion, T-maximax, selects any gamble that maximizes upper 
expectation E. The treatment of this criterion is very similar to that of T-maximax. Here, we 
simply mention that T-maximax satisfies subtree perfectness under exactly the same conditions 
as T-maximin. 



3.1.6. Maximality. Maximality starts from a strict partial order between gambles, conditional 
on an event A: 

(14) X> A Y whenever E(X - Y\A) > 

and then selects those gambles which are undominated with respect to this strict partial order: 

(15) m&xX := {X E X : VF E X, Y ^ A X} 

>-A 

for any finite set of gambles X. 

Here, one can show that, under strictly positive lower probability of conditioning events, 
maximality will satisfy subtree perfectness for trees of the type of Fig. Il(]ta)[ essentially because 

(16) E((AXi + A C Z) - (AXj + A C Z)\B) = E(A(Xi - Xj)\B). 

has the same sign as E(Xi — Xj\AC\B), because of the generalized Bayes rule [25], provided that 
E_(A\B) > 0. Note that we do not require marginal extension. 

We will see later that it does not satisfy subtree perfectness for trees of the type of Fig. IT^Tb) | 
In other words, maximality satisfies subtree perfectness only in some special cases. Inter- 
estingly, there are backward induction schemes which do work for maximality (under strictly 
positive lower probability) [TZl [51 [T31 [TP] . 
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Figure 17. A tree to demonstrate that subtree perfectness is not as strong as 
it may first appear. 




FIGURE 18. Solution of the tree of Fig. [17] 



3.1.7. E- admissibility. E-admissibility, selects any gamble that maximizes expectation under at 
least one probability distribution in Ai. 

Under strictly positive lower probability, as with maximality, it satisfies subtree perfectness 
for trees of the type of Fig. Ilffia)! It does not satisfy subtree perfectness for trees of the type of 
Fig-Efb)] 

The next example uses E-admissibility to demonstrate that subtree perfectness is not as strong 
a condition as it may first appear. 

Example 6. Consider the tree in Fig. [7?| Suppose that A4 = {Pi, P 2 }, with Pi(Ai) — P\(A 2 ) — 
P2(Ax) = P 2 (A 2 ) = 0.5, = Pi(B!\A 2 ) = 0.6, and P 2 (B 1 \A 1 ) = P 2 (B 1 \A 2 ) = 0.4. 

The normal form solution of the full tree is depicted in Fig. \18l The left-hand solution maxi- 
mizes expectation under Pi, whilst the right-hand solution maximizes expectation under P 2 . Of 
the four strategies, two are optimal according to E-admissibility. 

The normal form solutions of the subtrees at Ni and N 2 are depicted in Fig. I iff I In both 
subtrees, there are two possible strategies, and both are optimal according to E-admissibility. 

Subtree perfectness holds here, but observe that only two of the four global strategies are opti- 
mal, whilst all of the local strategies are optimal. 
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(B) 

FIGURE 19. Solutions of the subtrees at Ni and N 2 , for the tree of Fig. [T71 




Figure 20. An example involving imprecise utility. 

This shows that subtree perfectness is weaker than what one might call normal- extensive 
form equivalence. For further discussion of the relationship between subtree perfectness and 
normal-extensive form equivalence, we refer to [5]. 

3.2. Failures of Subtree Perfectness. In Section |2~T1 we already visited one well-known in- 
stance subtree perfectness failing, namely expected utility, when events of zero probability are 
involved. In this section, we provide many more examples. 

3.2.1. Imprecise Utility. 

Example 7. Consider again the very first example from the introduction (Fig. Q]), redrawn in 
Fig. \2(k note that this tree is an extremely simple instance of the tree in Fig. \l(ftb)\ which plays 
a central role in our subtree perfectness theorem. 

Suppose we are unsure about our utility between the three available options, and we are happy 
to accept any option as optimal that maximizes either of the following two utility functions: 

(17) C/i(n) = 3, EMr 2 ) = l, tfi(r 3 )=2, 

(18) U 2 (n) = -3, U 2 {r 2 ) = l, U 2 (r 3 ) = 2. 



Clearly, for the full tree, the optimal choice is {rx,ra}. However, for the subtree at N 2 , the 
optimal choice is {ri,r 2 }. Subtree perfectness is violated, because r 2 is not optimal in the full 
tree, whereas it is optimal for the subtree at N 2 . 
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Figure 21. An example involving a partial order. 




Figure 22. Failure of subtree perfectness for E- admissibility. 




Figure 23. E-admissible normal form solution of the tree of Fig. 

3.2.2. Choice Functions Not Induced By A Total Preorder. The previous example easily adapts 
to choice functions induced by a strict partial order. Indeed, assume that we have a strict partial 
order >- on gambles, and three gambles, X\, X 2 , and A3, such that: 

(19) X 3 >- X 2 , 
but where all other pairs are incomparable: 

(20) X x i- X 2 , X x f X 3 , 

(21) X 2 i- X u X 3 i- X x . 

Considering now the situation depicted in Fig. 1211 we are in exactly the same situation as 
before: clearly, for the full tree, the optimal choice is {X\, A3}. However, for the subtree at N 2 , 
the optimal choice is {Xi,X 2 }. Subtree perfectness is violated, because X 2 is dominated, and 
whence, not optimal, in the full tree, whereas it is optimal for the subtree at N 2 . 

Although this is not a rigorous proof, it does demonstrate why any non-trivial partial order 
might fail subtree perfectness. In other words, maximality (and also point-wise dominance and 
interval dominance, which we have not discussed here, but which are also induced by partial 
orders and are commonly used in the literature [25]) will fail subtree perfectness in general. 

Although E-admissibility does not correspond to a partial order, unsurprisingly, it fails subtree 
perfectness in a very similar way. 
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Figure 24. E-admissible normal form solution of the subtree at N%, for the 
tree of Fig. [22j 




Figure 25. Failure of subtree perfectness for maximin. 



Example 8. Consider the decision tree in Fia. \22l Suppose that M. = {Pi, P2], where P\(A\) — 
^2(^-2) = 0.8. The expectation of N3 is 3 under Pi and —3 under Pi. The remaining two 
strategies have constant values, namely 1, and 2. 

In the full tree, clearly, of the three strategies, only two are E-admissible, as depicted in 
Fia. \23l However, for the subtree at N%, both strategies are optimal, as depicted Fig. \24\ Subtree 
perfectness is violated, because the strategy with constant outcome 1 is dominated by the strategy 
with constant outcome 2, and whence, 1 is not optimal in the full tree, whereas it is optimal for 
the subtree at N2. In fact, we have mimicked the situation of Example^ 

3.2.3. Maximin. 

Example 9. Consider the tree in Fig. \25[ note that this tree is an extremely simple instance of 
the tree in Fig. \l^Kaj\ which plays a central role in our subtree perfectness theorem. 

Obviously, there are only two strategies. In the full tree, the worst possible outcome for both 
strategies is 0, and therefore both strategies are maximin. However, clearly, at N2, only the 
strategy yielding 2 is optimal. Whence, subtree perfectness is violated. 

3.2.4. r -maximin. When Eq. @ does not hold, r-maximin often fails to be subtree perfect, in 
a similar way that maximin fails it. 

Example 10. Consider the tree in Fig. \2b\ again note that this tree is a simple instance of the 
tree in Fig. \lf§fa)\ which plays a central role in our subtree perfectness theorem. 
Suppose that A4 = {Pi,P2\ with 



Pi 


Ai 


A 2 


Pi 


Ai 


A 2 


Bi 


0.1 


0.1 


and E>i 


0.4 


0.4 


B-2 


0.4 


0.4 


B 2 


0.1 


0.1 



The two strategies of the full tree are depicted in Fig. \'21\ and correspond to the following 
gambles: 





Ai 


A 2 




Ai A 2 


Bi 


2 


-1 


and Bi 


-1 


B 2 


-1 


2 


B 2 


2 
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Figure 26. Failure of subtree perfectness for r-maximin. 




FIGURE 27. The two strategies for the tree of Fig. [551 Only the left-hand 
strategy is r-maximin optimal. 








FIGURE 28. The two strategies for the subtree at Ni, for the tree of Fig. (55] 
Only the right-hand strategy is r-maximin optimal. 

The lower expectation of the first strategy is 0.5, whereas the second strategy has lower expectation 
—0.2. So, in the full tree, only the first strategy is optimal according to T -maximin. 
However, in the subtree at N± , the conditional probabilities are 





Pi(-I^i) 


P2(-\A 1 ) 


Bi 


0.2 and B 1 


0.8 


B 2 


0.8 B 2 


0.2 



and two strategies, depicted in Fig. \28l of this subtree, correspond to the following gambles: 
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B l 2 
B 2 - 



1 



and 



Bi 
B 2 



The lower expectation of the first strategy is —0.4, whereas the second strategy has lower expec- 
tation 0. So, in the subtree, only the second strategy is optimal according to T-maximin. 

Concluding, T-maximin does not satisfy subtree perfectness. In fact, the preference between 
the two strategies reverses depending on whether we consider the full tree, or only the subtree at 



We introduced the concept of subtree perfectness: in essence, we extended Selten's idea of 
subgame perfectness to resolute normal form solutions of decision trees. Subtree perfectness 
allows us to solve decision trees without worrying about which larger tree they are embedded 
in. This is a particularly useful property if one desires to avoid counterfactual reasoning for 
philosophical reasons. It also allows for efficient computational algorithms to solve decision trees 
via backward induction, and is tremendously helpful when something unexpected happens in 
the middle of a sequential decision problem, as subtree perfectness allows us to restrict our 
investigation to a subtree of the larger problem. 

We saw that, when using choice functions on gambles, subtree perfectness for all decision trees 
is equivalent to subtree perfectness for two simple classes of decision trees. Numerous examples 
for popular choice functions, demonstrated how this result can be leveraged both for proving 
subtree perfectness and for finding simple counterexamples. The application of the results to 
counterexamples is particularly useful, since it shows that, when searching for a counterexample, 
there is no reason to try complicated trees because a small one will always suffice. 

For choice functions that are not total preorders (that is, those that attempt to model indeter- 
minacy or indecision), we highlight a few important consequences. First, such choice functions 
always fail subtree perfectness in general; see the examples in Sections 13.2.11 and 13.2.21 and [9J 
Lemmas 15 & 21] for a full proof. We demonstrated that these counterexamples typically appear 
when the tree contains paths with more than one decision node. However, and this may come as 
a surprise, such choice functions can satisfy subtree perfectness for decision trees where a chance 
node is followed by single decision nodes; see the examples in Sections I3.1.4ft3~l~71 Most exam- 
ples of subtree perfectness in the literature have been total preorders failing subtree perfectness 
in exactly these trees, which may have created the impression that only expected utility can be 
used in such problems, and it may not be so well-known that alternatives such as partial orders 
can be useful here too. Problems in which one will observe some data and then make a single 
decision are not rare. Therefore, it is worth knowing that, in such cases, expected utility does 
not have a monopoly on subtree perfectness. 

In this paper we have only considered a subset of trees of this type: those in Fig. [T^a)| and 
the one in Fig. [TTJ We can consider a wider class of trees of this type: those with any size of 
partition at the first chance node, with each chance arc leading to a decision node involving any 
number of gambles. We could then try to find conditions for subtree perfectness in all trees of 
this type, and to find how one has to restrict particular choice functions to attain this behavior 
(for instance, an extension of Example [5]). Some preliminary results can be found in [7]. 

Subtree perfectness has strong links with several other properties of the solutions of sequential 
decision problems. The first is the classical method of backward induction: solving decision trees 
by "rolling back". This algorithm has not often been employed for criteria other than total 
preorders, but two similar algorithms have been proposed by Huntley and Troffaes [TU] and 
Kikuti et al. |13j . In particular the latter reference contains all the information required to find 



Ni. 



We refer to Seidenfeld 22 for further beautiful examples. 



4. Conclusion 
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the normal form solution for maximality and E-admissibility via backward induction. We can ask 
when backward induction gives the same solution as the normal form solution we have defined 
in this paper. As both the above works note, it is possible for backward induction to work but 
subtree perfectness to fail. Again, this typically never occurs when using a total preorder, so 
this result may not be well known. It is, however, quite easy to show that subtree perfectness 
implies backward induction [9J Corollary 27]. 

Another related question is the equivalence of our normal form solutions with extensive form 
solutions. In extensive form solutions, the subject makes decisions only upon reaching a decision 
node, rather than making all decisions ahead of time. Such a solution can be seen as a subtree of 
the initial tree, with some decision arcs removed. Upon reaching a decision node, the subject then 
simply picks one of the arcs that remains. Since subtree perfectness refers to consistency between 
local and global solutions, one might expect it to be sufficient for the equivalence between the two 
forms, but consider Example H3 Here, subtree perfectness is satisfied, but there is no extensive 
form solution available that we could really call equivalent. This behavior arises because there 
are many more possible normal form solutions than there are extensive form solutions. This 
topic requires further study; introductory material can be found in [51 § 7.2]. 
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